The widespread use of digital cameras has lead to an interest in new types of applications based on digital images. No longer are photographs just taken and stored for private purposes, but with the ubiquitous availability of the Internet digital images are shared among users in large databases of visual data, most notably community photo collections such as Flickr (http://www.flickr.com). These pictorial data collections contain vast amounts of high-quality images, often labeled with keywords or tags. Furthermore, digital images may be transmitted as search criteria in information queries, for example queries for information related to objects in a museum or to tourist attractions in a city. Despite the enormous quantity of image data that is publicly available on the Internet, and although there has been significant progress in image recognition capabilities, both for specific objects and for object classes, there is still a need for pictorial reference databases suitable for such pictorial query applications. Typically, the textual (annotations) and/or geographic (geo-tag) metadata associated with images of public collections are of far lower quality than their counterparts in “traditional” databases, such as stock photography or news archives. Thus, although there would be an abundance of pictorial data available publicly, the indexing information and other metadata associated with the pictorial data is not suitable for use in a reference database. As the metadata is inconsistent, inaccurate and/or incomplete, images with a related pictorial content cannot be associated with each other on the basis of their metadata. However, because of the enormous quantity of images involved, it is not efficient (too time-consuming) to establish the association of images with related pictorial content based solely on image matching capabilities, as every possible combination of two images would have to be processed.